Skip to content

feat(controller): support serverless serving with keda support by k8s scale subresource. #500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(controller): support serverless serving with keda support by k8s scale subresource. #500

wants to merge 4 commits into from
+186 −2

Conversation

@X1aoZEOuO
Copy link
Contributor

What this PR does / why we need it

Detailed Explanation of Commit

This commit introduces a guide for configuring serverless environments on Kubernetes, focusing on integrating Prometheus for monitoring and KEDA for autoscaling. The guide aims to optimize resource efficiency through event-driven scaling while maintaining observability for AI/ML workloads.

  • Prometheus Integration: Configured with namespaceSelector for cross-namespace monitoring
  • KEDA Autoscaling: Custom metric scaling with Prometheus triggers
  • Scale-to-Zero: Activator pattern with request buffering and CloudEvents

Which issue(s) this PR fixes

Fixes #

Special notes for your reviewer

Does this PR introduce a user-facing change?

cc @pacoxu @kerthcet

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Sep 28, 2025
@X1aoZEOuO X1aoZEOuO force-pushed the feat/1-n-keda-support branch from fea7121 to b73e6f0 Compare September 28, 2025 12:18
@X1aoZEOuO
Copy link
Contributor Author

/kind feature

@InftyAI-Agent InftyAI-Agent added feature Categorizes issue or PR as related to a new feature. and removed do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Sep 28, 2025
@X1aoZEOuO
Copy link
Contributor Author

X1aoZEOuO commented Sep 29, 2025 •
edited
Loading

@pacoxu @googs1025 @carlory @kerthcet Hello all! Could you spare a few minutes to review my PRs when you have a chance?

Other ref PRs:

@pacoxu
Copy link
Contributor

pacoxu commented Oct 9, 2025

/assign
I will take a look this week or early next week.

@X1aoZEOuO
Copy link
Contributor Author

@pacoxu @kenwoodjw Friendly ping, do you have some time to take a look at my PRs? Thanks a lot for your assistance!

@kerthcet
Copy link
Member

/assign

@pacoxu pacoxu mentioned this pull request Oct 28, 2025
Open
kerthcet
kerthcet reviewed Oct 28, 2025
View reviewed changes
Copy link
Member

@kerthcet kerthcet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems some docs are duplicated with https://github.com/InftyAI/llmaz/pull/499/files, can we just put one here and refer to it in another one.

@X1aoZEOuO
Copy link
Contributor Author

seems some docs are duplicated with https://github.com/InftyAI/llmaz/pull/499/files, can we just put one here and refer to it in another one.

@kerthcet Thank you for catching this! I've refactored the documentation structure to eliminate duplication, Now focuses specifically, and reference link to the main serverless documentation (PR #499)

@X1aoZEOuO X1aoZEOuO force-pushed the feat/1-n-keda-support branch from 6ff79f5 to 9b47451 Compare October 29, 2025 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kerthcet kerthcet kerthcet left review comments

@googs1025 googs1025 Awaiting requested review from googs1025

@carlory carlory Awaiting requested review from carlory

Assignees

@pacoxu pacoxu

@kerthcet kerthcet

Labels

feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@X1aoZEOuO @pacoxu @kerthcet @InftyAI-Agent